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Abstract 


The  performance  of  a  synthetic  discriminant  function-based  target-detection 
algorithm  is  detailed  as  the  filter  source  attributes  are  varied  to  reflect  var¬ 
ious  practical  aspects  of  filter  creation. 
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1.  Introduction 


This  is  the  third  report  in  a  continuing  series  of  reports  [1,2]  on  the  appli¬ 
cation  of  the  synthetic  discriminant  function  (SDF)  approach  to  automatic 
target  recognition  (ATR)  (and  the  exploration  of  related  topics).  This  report 
demonstrates  the  performance  of  the  SDF  algorithm  in  a  target-detection 
mode  as  the  SDF  filter  image  sources  progressively  transition  from  ideal 
sources  to  those  that  represent  a  more  real-world  scenario.  A  single  filter  is 
used  with  all  SDF  models.  This  use  of  a  single  filter  is  consistent  with  the 
optimum  performance  observed  and  detailed  in  the  first  report  [1]. 

To  test  the  SDF  performance,  I  used  the  standard  scene  test  set  sequence 
L1816S — rl.bin.  Each  of  the  236  scenes  in  this  sequence  contained  two  tar¬ 
gets:  M60  and  tnk.  For  this  study,  the  scenes  were  treated  as  if  they  con¬ 
tained  but  a  single  target.  That  is,  if  an  M60  filter  was  created,  then  all  scene 
returns  about  the  location  of  tnk  were  suppressed  and  counted  as  neither 
target  hits  nor  background  false  alarms.  In  this  way,  the  performance  of 
the  SDF  model  for  a  comparatively  easy-to-detect  target  (M60)  and  a  much 
more  difficult-to-detect  target  (tnk)  is  separately  measured.  This  approach 
is  also  consistent  with  the  way  one  would  ultimately  like  to  use  the  SDF 
model,  the  assumption  being  that  it  will  be  used  in  a  comparatively  target- 
poor  environment. 
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2.  Methodology  and  Results 


As  noted  in  the  introduction,  this  report  demonstrates  SDF  model  perform¬ 
ance  as  the  filter  image  source  progressively  transitions  to  more  realistic 
sources.  Because  the  SDF  model  is  not  scale  invariant,  scale  invariance  must 
be  built  into  the  filter.  This  is  done  by  superimposing  target  images  of  vary¬ 
ing  sizes  via  the  filter  creation  mechanism.  A  simple  mathematical  function 
has  been  defined  for  the  236  scenes  of  the  test  set.  This  function  permits  a 
fairly  tight  square  box  to  be  fitted  about  all  targets.  The  targets  can  as  a 
consequence  be  extracted  and  used  as  needed  for  creating  filters. 

The  boxes  are  also  used  to  define  target  hits.  Two  circles  are  fitted  to  these 
boxes:  an  inner  circle  (I)  just  touching  the  center  of  each  face  and  an  outer 
circle  (O)  touching  each  vertex.  (The  letters  indicated  within  the  parenthe¬ 
ses  form  a  simple  code  for  defining  test  condition  attributes  when  the  re¬ 
sults  are  presented.)  A  target  is  detected  if  a  hit  occurs  within  one  of  these 
circles.  As  noted  in  the  previous  report  [1],  two  options  can  be  used  to  de¬ 
fine  the  centroid  of  a  potential  target  hit.  The  first  is  to  define  the  hit  lo¬ 
cation  as  the  location  of  the  largest  peak  response  of  the  SDF  model  (N) 
within  the  area  of  one  of  the  previously  defined  circles  (discarding  all  other 
within-circle  peaks).  The  second  is  to  define  the  hit  location  as  the  peak 
value  weighted  average  of  all  hit  locations  within  the  circle  (Y).  Because 
the  previous  report  [1]  indicated  that  the  optimum  detection  filter  count  is 
one,  I  used  only  one  filter  to  present  the  results  of  this  report.  Three  models 
are  to  be  used  for  this  study:  a  baseline,  a  down-sampled,  and  an  all-aspect 
model.  Each  represents  a  different  approach  to  creating  SDF  target  filters. 

2.1  Baseline  Model 

A  filter  for  the  baseline  model  (B)  is  created  from  several  of  the  target  im¬ 
ages  contained  in  the  scene  set  used  to  test  the  performance  of  the  SDF  algo¬ 
rithm.  It  is  called  a  baseline  model  because  it  can  be  expected  to  outperform 
the  remaining  two  models.  This  expected  performance  is  a  consequence  of 
the  unrealistic  way  the  filter  is  created.  Nevertheless,  as  the  model's  name 
implies,  it  represents  a  good  baseline  for  comparing  the  performance  of  the 
remaining  models. 
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2.2  Down-Sampled  Model 

As  with  the  baseline  model,  a  filter  for  the  down-sampled  model  (D) 
is  created  from  the  scene  test  set.  The  sequence  of  test  scenes  contains 
targets  with  a  fixed  aspect  angle  but  varying  range.  This  is  demonstrated  in 
figure  1,  a  montage  of  four  scenes  taken  at  intervals  along  the  sequence  of 
test  images.  A  single  target  image,  the  largest  in  the  test  scene  sequence,  is 
selected  to  create  the  filter.  This  target  image  is  down  sampled  to  produce 
a  sequence  of  progressively  smaller  images  spanning  the  target-size  range 
in  the  scene  set.  This  sequence  of  down-sampled  images  is  used  to  create 
the  filter. 

2.3  All-Aspect  Model 

The  all-aspect  model  (A)  constructs  a  filter  from  representations  of  all  as¬ 
pect  angles  of  a  target.  The  filter  becomes  a  composite  of  both  these  all¬ 
aspect  angle  images  and,  because  scale  invariance  is  required,  down-sampled 
representations  of  these  images.  The  source  of  these  target  images  is  not 
the  set  of  test  scenes  (since,  among  other  reasons,  only  a  single  aspect  an¬ 
gle  is  available).  The  source  is  an  alternate  data  set  available  within  the 


Figure  1.  Example  scenes 
with  targets  indicated  by 
crosshairs. 
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U.S.  Army  Research  Laboratory  (ARL).  Although  the  scene  test  set  con¬ 
tains  but  a  single  fixed  aspect  angle  for  all  targets,  the  all-aspect  model 
represents  the  most  realistic  of  the  three  tests  of  the  SDF  algorithm.  This  is 
because  the  all-aspect  model  most  closely  represents  the  manner  in  which 
an  SDF-based  target-detection  algorithm  is  used  in  a  real-world  scenario. 


2.4  Results 


In  the  following  subsection,  I  present  the  results  of  the  three  filter  models 
as  a  series  of  target  detection  versus  false  alarm  curves.  The  first  series  is 
for  the  baseline  model.  Figure  2  contains  detector  performance  for  the  M60 
as  a  function  of  target  filter  image  count. 

The  images  were  taken  at  uniform  intervals  along  the  sequence  of  236  scenes 
The  target  image  count  for  the  corresponding  filter  is  given  in  the  integer 
column  of  the  legend.  The  remaining  legend  columns  are  self-explanatory. 
For  the  conditions  of  this  test,  the  optimum  filter  for  the  M60  is  created  from 
eight  target  images.  The  corresponding  results  for  the  tnk  are  also  given 
in  figure  2.  For  the  more  difficult  tnk  imagery,  the  optimum  filter  is  ap¬ 
proached  more  or  less  asymptotically  with  increasing  target  image  count. 


Figure  2.  (a)  M60  and 
(b)  tnk  baseline  (B) 
performance. 


O  =  Outer  circle  of  a  target  box 

Y  =  Hit  location  that  is  average  peak  value  of  all  hits  within  circle  peak  responses 
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Figure  3  gives  the  results  for  the  optimum  filters  of  figure  2  as  a  function  of 
the  remaining  variables  as  indicated  in  the  legend. 

No  clear  guideline  emerges  from  this  study  with  regard  to  the  relation¬ 
ship  between  performance  and  the  variables  examined.  The  results  for  the 
down-sampled  model  are  given  in  figure  4. 

Two  down-sampled  schedules  were  used.  The  first  schedule  produced  a 
sequence  of  11  images  with  each  2  x  2  pixels  smaller  than  the  previous 
image.  The  second  schedule  produced  a  sequence  of  five  images  with  a  4 
x  4  down  sample  between  images.  The  starting  image  was  number  275,  an 
image  near  the  end  of  the  test  sequence.  The  performance  of  both  the  M60 
and  tnk  fell  when  compared  with  the  baseline  models.  The  performance 
fall  was  especially  severe  for  the  tnk,  where  the  false-alarm  rate  increased 
by  an  order  of  magnitude  for  the  same  target-detection  rate. 

To  create  suitable  filters  for  the  all  aspect  model,  I  first  examined  a  rather 
extensive  U.S.  Army  Aviation  and  Missile  Command  (AMCOM)-provided 
image  set.  The  available  files  are  listed  in  the  appendix.  This  list  excludes 
the  standard  SDF  test  sequence.  These  files  contain  approximately  30,000 


Figure  3.  Additional 
(a)  M60  and  (b)  tnk 
baseline  (B)  performance 
curves. 


O  =  Outer  circle  of  a  target  box 

Y  =  Hit  location  that  is  average  peak  value  of  all  hits  within  circle  peak  responses 
I  =  Inner  circle  of  target  box 

N  =  Hit  location  of  largest  peak  response  of  SDF  model 
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Figure  4.  M60  (lower  pair 
of  curves)  and  tnk 
down-sampled  (D) 
performance  curves. 
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target-containing  scenes.  Both  the  lack  of  ground-truth  data  and  the  quality 
of  the  imagery  precluded  the  use  of  the  image  set  as  a  filter  source. 

A  second  potential  all-aspect  filter  image  source  (and  the  one  that  was  used 
in  this  study)  is  a  forward-looking  infrared  (FLIR)  database  generated  by 
the  Army's  Night  Vision  and  Electronic  Sensors  Directorate  (NVESD).  This 
database  has  the  desirable  attribute  that  much  of  the  target  imagery  is  avail¬ 
able  for  all  aspect  angles  (at  5°  increments).  To  achieve  a  best  match  be¬ 
tween  the  best  NVESD  M60  and  tnk  sequences  and  the  AMCOM  SDF  im¬ 
ages,  I  adjusted  the  intensity  scales,  and  for  the  M60, 1  modified  the  target 
images  themselves.  The  resultant  NVESD  sequences  (40  x  75  pixels)  are 
given  in  figures  5  and  6. 

The  M60  modifications  should  be  apparent  in  figure  5.  Figure  7  is  scene 
275  of  the  SDF  test  set  sequence  with  the  5°  aspect  angle  NVESD  images 
superimposed  (images  not  to  scale). 

I  performed  experiments  to  find  the  best  down-sampling  schedule  for  the 
all-aspect  model.  For  the  5°  aspect  angle  M60  image,  the  following  se¬ 
quence  was  found  to  produce  the  best  filter  model:  2,  4,  6,. .  .28.  The  num¬ 
bers  refer  to  the  factor  by  which  the  x  and  y  pixel  count  was  reduced  for 
the  original  40  x  75  windowed  image  (as  given  by  the  first  image  of  figure 
5).  The  same  filter  sequence  was  used  to  produce  filters  for  all  the  aspect 
angles  of  the  M60  from  5°  through  180°.  Each  filter  was  tested  on  the  stan¬ 
dard  sequence  of  images  (images  with  an  M60  and  tnk  aspect  of  about  0°  to 
5°)  with  the  results  of  the  tests  given  in  figure  8.  These  curves  are  the  M60 
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Figure  5.  All-aspect-angle 
sequence  of  M60:  5° 
through  180°  (40  x  75 
pixels). 


Figure  6.  All-aspect-angle 
sequence  of  tnk:  5° 
through  180°. 


Figure  7.  SDF  test  set 
sequence  scene  275  with 
superimposed  NVESD 
images. 


detection  rates  for  average  false-alarm  rates  of  0.5, 1.0,  and  2.0.  Unexpect¬ 
edly,  the  performance  of  these  curves  at  180°  is  better  than  at  5°.  While  the 
180°  view  of  the  M60  has  approximately  the  same  silhouette  as  the  front 
view,  the  gray-scale  appearance  of  the  M60  at  180°  does  not  look  like  the 
target  presentation  in  the  test  scenes.  Figure  8  also  shows  the  results  for  tnk. 
Despite  the  seemingly  good  match  at  the  0°  to  5°  aspect  angle  between  the 
NVESD  and  AMCOM  tnk  images,  the  performance  was  quite  poor. 

A  final  test  was  performed  in  which  the  filter  was  composed  of  images 
at  all  scales  and  all  aspect  angles.  Four  curves  are  generated  for  M60  and 
tnk.  A  360°  representation  of  the  images  for  figures  5  and  6  was  created 
by  mirroring  of  the  images  below  180°.  These  images  were  sampled  at  5°, 
10°,  20°,  and  40°  increments.  The  results  are  given  in  figure  9.  Because  the 
angular  increments  show  little  difference  in  performance,  I  did  not  label 
the  individual  curves. 
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3.  Conclusion 


The  performance  of  an  SDF-based  target-detection  algorithm  is  examined 
for  a  range  of  filter  image  sources.  Results  of  a  baseline  source  taken  from 
the  algorithm  test  set  are  compared  with  results  from  sources  that  repre¬ 
sent  more  of  a  realistic  operational  scenario.  The  results  demonstrate  the 
difficulty  of  maintaining  SDF  performance  as  filter  sources  transition  away 
from  unrealistic  image  sets. 
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Appendix.  AMCOM  Image  Files  and  ARL  Codes 


The  list  of  AMCOM-provided  FLIR  image  sets  noted  in  the  main  body  of 
the  report  are  given  below: 

ml.t38.110143.lA040.seqb 

ml_t39_110215_lA713.seqb 

m3.tl3a_072640.4C899.seqb 

m3-tl4a.072738.4D515.seqb 

m3.tl5.073100_4F140.seqb 

m3.tl6_073159_4FDC0.seqb 

m3.tl6a-073138.4F94E.seqb 

m3.t53_0612152232Fl.seqb 

m3.t56_06113022294F.seqb 

m3_t65_074805.5A12E.seqb 

m3.t74_075225_5D89C.seqb 

m3.t85.080812.6998D.seqb 

m3_t94_080840.69EFC.seqb 

m4_144120.5e35.seqb 

m4_150426.e8b2.seqb 

m5.tll.12.133825.dd50.seqb 

m5_tl_132143.2B4D.seqb 

m5.t2_13225323A39.seqb 

m5.t3.132446_5251.seqb 

m5.t4_132533_5C7B.seqb 

m5.t5.132956_70AD.seqb 

m5_t6_133054.7D0E.seqb 

m5.t7.133335_9F62.seqb 

m5_t9_10_133610_C069.seqb 

m8_tl0_194316.241ED.seqb 

m8.tl2_194450_25605.seqb 

m8_tl3_194840.28359.seqb 

m8_tl4_194920.28C0A.seqb 

m8.tl6_195040_29D18.seqb 

m8_tl8_200130.seqb 

m8_tl9_201932_4019.seqb 

m8_t21.202300_6CBD.seqb 

m8.t282204210_1624D.seqb 

m8_t292204250_16AEl  .seqb 


13 


m8J8.193833_22256.seqb 
m8.t8a_l  93915 _22B5F.seqb 
m8  J9.1 94240 .  seqb 
m8Ja.185500.C4B0.seqb 
m8Jb_185540.CD73.seqb 
m9_135251_8963.seqb 
m9_135600.b5d2.seqb 
m9_143026.9b05.seqb 


The  following  is  a  list  of  all  codes  including  brief  descriptions  that  I  devel¬ 
oped  for  this  study  (sdf.c:  modified*): 


aspect_angle.m60.c: 

aspect.angle.tnk.c: 
convert.c: 
display  Jile.c: 
display  Jilel.c: 
filter  .cluster.c: 
filter.clusterl.c: 


flip.chip.c: 
makeJilter  Jmage.c: 
makeJrameJist.c: 

makeJist.c: 

make_nvlJist.c: 

make_sceneJist.c: 

montage.c: 


Reads  one  of  the  outputs  of  sdf.evaluate.c:  results.%d.dat  and  ex¬ 
tracts  information  for  MATLAB  plots. 

See  aspect_angle.m60.c. 

Converts  the  output  of  montage.c  into  an  sdf.c  usable  format. 

Reads  and  displays  contents  of  an  image  file. 

Reads  and  displays  contents  of  an  image  file. 

Clusters  to  generate  detection  filters  for  use  by  sdf.c. 

Same  as  filter  .cluster.c  but  modified  to  allow  code  to  operate  on  a  sub¬ 
set  of  the  input  imagery  (see  main).  This  code  requires  the  existence 
of  file:  merged  Jile. 

Flips  40  x  75  NVESD  chips  to  make  a  full  360°  SDF  target  presentation. 

Creates  SDF  target  filter  images  from  the  standard  scene  test  set. 

Lists  the  frames  of  the  standard  scene  test  set  for  use  as  test.list  by 
sdf.c  (a.out  -td  1). 

Lists  masked  target  image  files  for  sdf.c  detector  filter  builder. 

Lists  test.list  for  sdf.c  from  NVESD  all  aspect  angle  40-  x  75-pixel 
images  (chips). 

Lists  (test.list)  the  SDF  test  set  scenes  as  input  to  the  filter  maker  in 
sdf.c. 

Creates  a  montage  of  sig  40-  x  40-chip  images  for  one  aspect  of  M60 
or  T72  and  includes  as  final  image  in  montage  the  corresponding  SDF 
target  image.  Also  creates  a  montage  of  all  aspect  angles  for  a  selected 
NVESD  chip.  This  version  is  optimized  for  tnk. 


‘The  original  version  of  sdf.c  was  written  by  Lipchen  (Alex)  Chan  of  ARL. 
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montagel.c:  Is  same  as  montage.c  but  optimized  for  M60. 

movie.c:  Creates  MATLAB-compatible  image  file  to  test  ground-truth  values 
in  movie  mode. 

movie_maker.c:  Reads  and  displays  contents  of  a  file. 

plot_maker.c:  Creates  MATLAB  2-D  plots  for  SDF  filter  results. 

plot_makerl.c:  Is  same  as  plot-maker.c  but  allows  a  more  general  labeling  of  the  out¬ 
put  legends. 

read_filel.c:  Reads  and  displays  contents  of  a  file. 
read_test.dfil.c:  Reads  and  displays  contents  of  test.dfil. 

sdf.c:  Is  the  modified  version  of  sims_sdf.c,  the  implementation  of  the  SDF- 
based  ATR  algorithm. 

sdf_evaluatel.c:  Evaluates  output  of  sdf.c:  detection  rate  versus  false-alarm  rate.  This 
version  expects  the  first  entry  from  sdf_output.dat  to  be  scene  ID. 
Scene  ID  is  used  to  adjust  the  value  of  edge  to  either  an  inner  box 
circle  or  an  outer  box  circle. 

sdf_imaging.c:  Reads  and  displays  in  MATLAB  format  the  detector  images  outputted 
by  sdf.c.  Images  contain  target  ground  truths.  Also  has  option  of  read¬ 
ing  a  set  of  the  original  scenes. 

templates:  Is  a  target-tracking  code, 
temptape.c:  Reads  and  displays  contents  of  an  image  file. 

test_gt.c:  Creates  a  MATLAB-compatible  image  file  to  test  ground-truth  values, 
view.c:  Views  128  x  128  SDF  scene  with  superimposed  target  images. 
view_aspect.c:  Creates  M60  and  tnk  (5°  through  180°  aspect  angle)  image  montage. 
view_aspectl.c:  Creates  M60  and  tnk  (0°  through  355°  aspect  angle)  image  montage. 

Source  code  listings  for  these  codes  are  available  from  the  author  upon 
request. 

The  core  codes  and  their  execution  sequence  for  the  data  generated  for  the 
aspect  angle  models  are  given  below: 

•  make_nvl _list.c  j 

•  sdf.c  — ►  a.out  -bd  1  [ 

•  make_frame  Jist.c  | 

•  sdf.c  — >  a.out  -td  1  J. 

•  sdLevaluatel.c 
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